Q-Batch: initial results with a novel update rule for Batch Reinforcement Learning

Batch Reinforcement Learning has established itself as a valuable alternative to develop learning and adaptive agents. Batch Reinforcement Learning algorithms are characterized by obtaining a policy from a set of collected data. Common methods apply adapted versions of RL update rules, such as QLearning, on the transitions of the batch, building a pattern set. The target values of the pattern represent a value function, which is latter “fitted” with a function approximator using batch supervised learning methods. This paper presents the first results with a novel update rule, Q-Batch. The proposed method is benchmarked against the batch version of Q-Learning and Watkins Q(λ) in the Neural Fitted Q Iteration framework. The proposed work is tested in the Predator-Prey simulated environment. Empirical results show that the proposed method is able to achieve comparable or better asymptotical performance while requiring fewer interactions with the environment.